Sorting and Selection with Random Costs 

Stanislav Angelov * Keshav Kunal ^ Andrew McGregor ^ 

February 2, 2008 



Abstract 

There is a growing body of work on sorting and selection in models other than the unit- 
cost comparison model. This work is the first treatment of a natural stochastic variant of the 
problem where the cost of comparing two elements is a random variable. Each cost is chosen 
independently and is known to the algorithm. In particular we consider the following three 
models: each cost is chosen uniformly in the range [0, 1], each cost is with some probability 
p and 1 otherwise, or each cost is 1 with probability p and infinite otherwise. We present 
lower and upper bounds (optimal in most cases) for these problems. We obtain our upper 
bounds by carefully designing algorithms to ensure that the costs incurred at various stages 
are independent and using properties of random partial orders when appropriate. 

1 Introduction 

In the relatively recent area of priced information [SI [SI |3] , there is a set of facts each of which can 
be revealed at some cost. The goal is to pay the least amount such that the revealed facts allow 
some inference to be made. A specific problem in this framework, posed by Charikar et al. is 
that of sorting and selection where each comparison has an associated cost. Here we are given a set 
y of 71 elements and the cost of comparing two elements u and v is C(^u,v)- This cost is known to 
the algorithm. We wish to design algorithms for sorting and selection that minimize the total cost 
of the comparisons performed. Results can be found in [141 dOl [H] where the performance of the 
algorithms is measured in terms of competitive analysis. In all cases assumptions are made about 
the edge costs, e.g., that there is an underlying monotone structure [T31[TD] or metric structure [TT] . 

A related problem that predates the study of priced information is the problem of sorting nuts 
and bolts [TJ [T^ . This is a problem that may be faced by "any disorganized carpenter who has a 
mixed pile of bolts and nuts and wants to find the corresponding pairs of bolts and nuts" according 
to the authors of [1]. The problem amounts to sorting two sets, X and Y, each with n elements 
given that comparisons are only allowed between u E X and v Cz Y. It can be shown that this 
problem can be generalized to the priced information problem in which comparison costs are either 
1 or oo. 

In this paper we study a natural stochastic variant of the sorting problem. We consider each 
comparison cost to be chosen independently at random. Specifically, we consider the following three 
models: 
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(a) Uniform Comparison Costs: C(^u.v) is chosen uniformly in the range [0, 1], 

(b) Boolean Comparison Costs: C(^u.v) = with probability p and 1 otherwise, and 

(c) Unit and Infinite Comparison Costs: C(^u,v) — 1 with probability p and oo otherwise. 

This first model is in the spirit of the work on calculating the expected cost of the minimum spanning 
tree [9] . The second and third models are related to the study of random partially ordered sets (see 
[3] for an overview) and linear extensions [SKI^IH]. Specifically, in Model (b), the free comparisons 
define a partial order (V, =4) that is chosen according to the random graph model. To sort V we 
need to do the minimum number of remaining comparisons to determine the linear extension, or 
total order, defined by the remaining comparisons. In Model (c) we have the problem of inferring 
properties of the random partial order {V, =:^) defined by the cost 1 edges. 

1.1 A Motivation from Game Theory 

The framework of priced information lends itself naturally to a game theoretic treatment where 
there are numerous sellers each owning one or more facts. Some facts will be, in a sense, more 
valuable than others. In the case of sorting, the value of a comparison is inversely related to 

\{w : u < w < V or V < w < u}\ because for each such w, the comparisons (u, w) and (w, v) together 
provide an alternative way of implying (u, v). How should sellers price their information in an effort 
to maximize their profit? Herein lies the dilemma — if the pricing of the facts is strictly monotonic 
with their value, the buyer can infer the sorted order from the prices themselves and by performing 
a single (cheapest) comparison! Yet, if there is no correlation, the seller is not capitalizing on the 
value of the information they have to sell. It seems likely that the optimum pricing of a fact will be 
a non-deterministic function of the value. While a treatment of the game theoretic problem seems 
beyond our reach at this time, we feel that a first step will be to find optimal buyer strategies when 
the price of each fact is chosen randomly and independently of the value of the fact. 

1.2 Our Results 

For p — 1/2, our results are summarized in Table [H In general, we will present bounds in terms 
of both n and p. Note that rather than using a competitive analysis of our algorithms (as in 
mmnmi]) we estimate the expected cost of our algorithms and the expected cost of the respective 
minimum certificate. 
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Table 1: Comparison between the expected costs of our algorithms and the minimum certificates 
for sorting and selection for various cost functions. The first row follows from standard algorithms 
and is given as a reference point for comparison. Also, in the case of C(^u,v) S {1; 00} we consider 
finding all maximal/minimal elements. 
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We would like to note that for the first three rows of Table [1] the expected cost of each compar- 
isons is 1/2 but the variance differs. For selection type problems the variance makes a big difference 
since there are many ways to certify the rank of an element. However for sorting there is only one 
(minimal) certificate for the sorted order. Nevertheless, a little bit of variance makes it possible to 
sort with only linear cost rather than O(nlogn) cost. 

One of the main challenges in the analysis of our algorithms is to ensure that the costs incurred 
at various stages of the algorithm are independent. We achieve this by carefully designing the algo- 
rithms and describing an alternative random process of cost assignment that we argue is equivalent 
to the original random process of cost assignment. 

2 Preliminaries 

We are given a set V oi n elements, drawn from some totally ordered set. We are also given a 
non-negative symmetric function c : V x V —> TV^ which determines the cost of comparing two 
elements of V . Given V and c, we are interested in designing algorithms for sorting and selection 
that minimize the total cost of the performed comparisons. 

The above setting is naturally described by the complete weighted graph on V, call it G, where 
the weight Ce of an edge e is determined by the cost function c. The direction of each edge (u, v) 
in G is consistent with the underlying total order and is unknown unless the edge e is probed^ i.e., 
the comparison between u and v is performed, or it is implied by transitivity, i.e., a directed path 
between u and v is already revealed. In this case we call u and v comparable. 

An algorithm for sorting or selection should reveal a certificate of the correctness of its output. 
In the case of sorting, the minimal certificate is unique, namely the Hamiltonian path in G between 
the largest and the smallest elements of V . In the case of selection, the certificate is a subgraph of 
G that includes a (single) directed path between the element of the desired rank and each of the 
remaining elements of V . In the special case of max-finding, the certificate is a rooted tree on V, 
the maximum element being the root. The cost of a certificate is defined as the total cost of the 
included edges. 

In this paper we consider three different stochastic models for determining the cost function c 
(see Section [T]). In Models (b) and (c), the graphs induced respectively by the cost or 1 edges have 
natural analogue to random graphs with parameter p, denoted by Gn,p- Note that in Models (a) 
and (b), the maximum cost of a comparison is 1. When this is case, the following proposition will 
be useful and follows from a natural greedy strategy to find the maximum element in the standard 
comparison model. 

Proposition 1. Given a set V of n elements, drawn from a totally ordered set, where the cost 
of the comparison between any two elements is at most 1, we can find (and certify) the maximum 
element performing n — 1 comparisons incurring a cost of at most n — 1. 

We will measure the performance of our algorithms by comparing the expected total cost of the 
edges probed with the expected cost of a minimum certificate. Note that the cost of the minimum 
certificate is concentrated around the mean in most cases. Even when the minimum certificate cost 
is far from the mean, we can obtain good bounds on the expected ratio by using algorithms from 
[4] (Model (a)) or standard algorithms (Model (b) and (c)). 

Finally, in the analysis, it would be often useful to number the elements oiV, vi, - ■ ■ ,w„ such 
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that vi < ■ ■ ■ < Vn- We also define the rank of an element v with respect to a set 5 C y to be 

rk(ti) — \{u : u < v,u ^ S}\ . 

3 Uniform Comparison Costs 

In this section we will assume that the cost of each comparison is chosen uniformly at random in 
the range [0, 1]. We consider the problems of finding the maximum or minimum elements, general 
selection, and sorting. The algorithms are presented in Fig. [T] 

Theorem 2. The expected cost o/ | UniformFindMax \ is at most 2(iJ„ — 1) where Hk ~ X]i=i 

Proof. We analyze a random process where we consider edges one by one in a non-decreasing order 
of their cost. Note that the costs of edges define a random permutation on the edges. If an edge is 
incident to two candidate elements, i.e., elements that have not lost so far a performed comparison, 
we probe the edge, otherwise we ignore the edge. Either way we say the edge is processed. 

We divide the analysis in rounds. A round terminates when an edge is probed. After the end 
of a round, the number of candidates for the maximum decreases by one. Therefore after n — 1 
rounds the last candidate would be the maximum element. For r G [n — 1], let t^ denote the random 
variable which counts the number of edges processed in the rth round. Let — X]i=i ^» denote 
the rank of the edge (in the sorted by costs order) found in the rth round. Therefore, the expected 
cost of the performed comparison is E [Tr] /{{^2) + 

It remains to show an upper bound on the value of E [T^] = X]i=i'^[^i]- Tr-i edges 

have been processed. The probability that the next edge is between two candidate elements is 
P=r^2~'^)/{G)-Tr-i) > (""^r'V(2)- Hence, for r G [n-2],E[tr] < 1/p < (^)/("-(r'^)' 
and for r = n — 1, we have E [Tr] < {^). We conclude that the total expected cost is at most, 

□ 

Theorem 3. The expected cost of the cheapest rank k certificate is + Hn-k+i — 2. 

Proof. Consider Vi with i < k. Any certificate must include a comparison with at least one of 
Wi+i, . . . ,Vk. The expected cost of the minimum of these k — i comparisons is ■ Summing 

over i, i < k, yields Hk — 1. Similarly, now consider Vi with i > k. Any certificate must include 
a comparison with at least one of Vk, . . . The expected cost of the minimum of these i — k 

comparisons is -jzrt+T- Summing over i, n > i > k, yields — 1. The theorem follows. □ 

Note that the theorem above also implies a lower bound of SI (log n) on the expected cost of the 
cheapest certificate for the maximum (minimum) element. To prove a bound on the performance 
of I Uniforms election\ we need the following preliminary lemma. 

Lemma 4. Let v Cz V and perform each comparison with probability p. Then, with probability at 
least 1 — 1 /n* ( assuming p > 1 /n^), for all u such that 

|rk.-rk.|>'^"^°^"^^°^" + ^°^^'/^^^ 
the relationship between u and v is certified by the comparisons performed. 
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Proof. W.l.o.g., rky w > n/2. Wc will consider elements in 5 = {u : rky u < rkvw}. The analysis 
for elements among {u : rky u > rky v} is identical and the result follows by the union bound. 
Throughout the proof we will assume that n is sufficiently large. 

Let D be the subset of S such that u € D is comparable to v. We partition S into sets, 



Bi = \u : Av — wi < rku < rkv — w(i — 1)} , 
V V V 



where w = ■) . Let Xi = D D Bi, that is, the elements from set Bi that are comparable to v. 

For the sake of notation, let Xq = {v}. Let Di = Uo<j<i-i -^j- If we perform a comparison between 
an element of Dt and an element u of Bi then we certify that u is less than v. The probability that 
an element of Bi gets compared to an element of Di is, 



1 - (1 -p)"' > l-e-P^^ > (1 -e-i)max{l,pA} • 

Let {Yi)i<i be a family of independent random variables distributed as 'Bin{w,q) where q 
e^i) max{pA, !}• Note that E [F,] = UD, logn if < 1- 

1. For i such that Di < 1/p. Using the Chernoff/Hoeffding Bounds, 

E[Y] 



(1 



'[X, < Alogn] 



< 



12 



1 



In other words, the number of comparable elements increases by at least a logn factor until 
Di > 1/p. Therefore, with probability at least 1 — log(l/p)/n^, for all i, 



A >min{l/p, (logn)*-^} 



In particular, Aogi/p ^ l/P- 



2. Assume that i > log(l/p) and therefore -Diogi/p ^ V-P- Using Chernoff/Hoeffding Bounds, 
we get 
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Therefore with probability at least 1 — Dt > where t = 61ogn + log(l/p). Consider 

an element u € Bt' where t' > t. The probability that u is not in D is bounded above by 
(1 - p)6iogn/p < i/„6_ Hence, with probability at least 1 - (1 + t)/n^, 



\S\D\<wt< 



150 logn (log 71 + log (1/p)) 



□ 



Lemma 5. Consider the algorithm \ UniformFindRankCertificate\ called on a randomly chosen v. 
With probability at least 1 — n~^ the algorithm returns a certificate for the rank of v. The expected 
cost of the comparisons is 0(log^ n). 
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Algorithm UniformFindMax{V) 

1. for j = 1 to n — 1 

2. do Perform cheapest remaining comparison 

3. Remove the smaller element of the performed comparison 

4. return remaining element 

Algorithm UniformFindRankCertificate{V, v) 

1. Let a = 1200 log^ n 

2. Perform all comparisons e such that Cg e [0, a/n] 

3. {or ueV 

4. do if u is comparable with v 

5. then V ^V\{u} 

6. it u<v then Vi ^ViU {u} else V2 ^ V2 U {u} 

7. for i = 1 to log2(n/a) 

8. do Perform all comparisons e such that Cg G [a2'^~^ /n,a2'^ /n] 

9. Repeat Steps 3-6 

10. return Vi, V2 

Algorithm UniformSelection{V, k) 

1. ii \V\ = 1 then return V 

2. Pick random pivot v E V 

3. (Vi, V2) H UniformFindRankCertificate[ V, v) 

4. T/^F\{u} 



5. if \Vi\ > k then \UniformS election Vi , k) else | UniformS election 'V2 , fc — |Vi|) 



Algorithm UniformS ort{V) 
1. Pick random pivot v G V 



2. {Vi, V2) ^ UniformFindRankCertificate 'V, v) 

3. return ( | [/n^/or7?^■6'orf[ VL)1 'i^; UniformS ori\ y2)) 



Figure 1: Algorithms for uniform comparison costs. 



Proof. Let Vi be the set of elements at the start of iteration i. Let pi = a/n be the probability that 
Ce e [0,Q;/n]. For i > 1, let pi — a2*~^/n be the probability that Ce G [a2*~-'^/ri, a2Y'T-]. First we 
show that, with probability at least 1 — ^"^"^"^"^ , for all 1 < i < log2(n/a), \Vi\ < n/2*~^. Assume 
that \Vi\ < n/2*~^. Appealing to LemmaSl there are less than 

300 log|^,|(log 1^,1 +log(l/p)) eOOlog^n _ 
p az' '-/n 

elements in V^+i \ Vi and hence the |Vi+i| < It remains to show that the cost per iteration is 

0(log^?i). This follows since the expected number of comparisons is 0{V^a2^ /n) = 0(an/2*) and 
each comparison costs at most a2^ /n. □ 

The following theorem can be proved using standard analysis of the appropriate recurrence 
relations and Lemma [5l 
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Algorithm BooleanFindMax{V) 

1. Perform all free comparisons 

2. Find the maximum element among the elements that have not lost a comparison 
in Step 1 using cost 1 comparisons. 

Algorithm BooleanSelection{V, k) 

1. Perform all free comparisons 

2. w ^ 3(logn)/p2 

3. S" <— {w : w wins at least k — I — w comparisons and loses at least n — k — w 
comparisons} 

4. Find the minimum and maximum element of S and determine their exact rank 
by comparing them to all elements whose relation to them is unknown. 

5. Tmin *— rky (minimum element of S) 

6. Tmax <— rky (maximum element of S) 

< rkv(v) < r^ax} 

8. if ^min k and Vmax ^ k 

9. then return Standards election{T, k — rmin) 

10. else return StandardSelection{V, k) 



Figure 2: Algorithms for boolean comparison costs 



Theorem 6. The algorithm \ UniformSelection\ can be used to select the kih element. The expected 
cost of the certificate is 0(log n). The algorithm \ UniformSort\ returns a sorting certificate with 
expected cost 0{n). 

Note that we can check if a certificate is a valid one without performing any additional com- 



parisons. In the case when UniformFindRankCertificate fails, we can reveal all edges to obtain a 



certificate without increasing asymptotically the overall expected cost. 

Theorem 7. The expected cost of the cheapest sorting certificate is {n — l)/2. 

Proof. For each 1 < i < n — 1 there must be a comparison between Vi and Ui+i. The expected cost 
of each is 1/2. The theorem follows by linearity of expectation. □ 



4 Boolean Comparison Costs 

In this section we assume that comparisons are for free with probability p and have cost 1 otherwise. 
We consider the problems of finding the maximum or minimum elements, general selection, and 
sorting. The algorithms for maximum finding and selection are presented in Fig. [51 For sorting we 
use results from [2] and [14j to obtain a bound on the number of comparisons needed to sort the 
random partial order defined by the free comparisons. 

Theorem 8. The expected cost of \BooleanFindMax \ is \/p ~ 1 as n oo. 

Proof. Consider the ith largest element. The probability that there is no free comparison to a larger 
element is (1 — Hence, after performing all the free comparisons, the expected number of 
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non-loscrs, in the limit as n tends to infinity, is 



lim V(l -p)*-! = lim ^^—EH. ^ i/p . 

Hence, by Proposition [l] the expected number of comparisons of cost 1 that are necessary is 
l/p-l. □ 

The theorem above leads to an immediate corollary: 

Corollary 9. The expected cost of the cheapest certificate for the maximum element and the element 
of rank k is D,{l/p) as n ^ oo. 

Using Theorem [51 we obtain a sorting algorithm with expected cost of at most {l/p— — 1) 
by repeating n — 1 times \BooleanFindMax\ We improve this result (for sufhciently small p) by 
observing that the free comparisons define a random partial order on the n elements, call it Gn,p- 
In [2] , the expected number of linear extensions of Gn,p was shown to be 



n 



i-ii-p)\ 1 



n n" 1 

fc=l ^ ^ 

A conjecture, independently proposed by Kislitsyn T^, Fredman |5], and Linial [T7], states that 
given a partial order P, there is a comparison between two elements such that the fraction of 
extensions of P where the first elements precedes the second one is between 1/3 and 2/3. Ignoring 
running time, this would imply sorting with cost \og^^2^{P): where e(P) denotes the number of 
linear extensions of P. In |13j , a weaker version of the conjecture was shown giving rise to an efficient, 
via randomization [7], sorting algorithm with cost logn^^ e{P). Taking a different approach, Kahn 
and Kim [12] described a deterministic polynomial time, 0(loge(P)) cost algorithm to sort any 
partial order P. 

Combining the above results, and using Jensen's inequality, we obtain a sorting algorithm with 
expected cost at most, 

logiA e(G„,p) < (logrL - 1) . 

Note that for p < 0.1389, \ogii/g,{l/p) < 1/p — 1. Combining the two sorting methods, we obtain 
the following theorem. 

Theorem 10. There is a sorting algorithm for the Boolean Comparison Model with expected cost 
o/min{logii l/p, 1/p — 1} • (n — 1). 

The proof of the following theorem about the cheapest sorting certificate is nearly identical to 
that of Theorem [71 

Theorem 11. The expected cost of the cheapest sorting certificate is (1 — p){n — 1). 

We next present our results for selection. 

Theorem 12. The alaorithm \BooleanSelection\ can he used to select the fcth element. The expected 
cost of the algorithm is 0{p^^\ogn). 
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Proof. We want to bound the size of set S as defined in the algorithm. Fix an element Vj. For an 
element Vi such that i < j , let I — j — i — Consider the event that we can infer Vi < Vj from the free 
comparisons because there exists an element Vi' such that Vi < Vi' < Vk and C(^vi,v-,) — '^{v^,,vj) = 0. 
The probability of this event is 1 — (1 — p^)' and hence with probability at least 1 — we learn 
Vi < Vj it I > w = 3(logn)/p^. Therefore, with probability at least 1 — l/n^, Vj wins at least 
j — 1 — w comparisons. Similarly with probability at least 1 — l/n^, Vj loses at least n ~ j + w 
comparisons. 

Hence, with probability 1 — every element from the set 

S' ^ \v : k — w < rkv < k + w} , 

V 

belongs to the set S and in particular the element of rank k also belongs to S. Note that no 
element from outside 5" can belong to S and hence \S\ < 2w. By Proposition [TJ it takes 0{w) 
comparisons to compute the minimum and maximum elements in 5*. There are at most 2w elements 
incomparable to the minimum (maximum) element with probability at least 1 — and hence the 
expected cost for determining the exact rank of minimum (maximum) element from S is bounded 

by 

2w(l - 2/^2) + (n - 1)2/^2 = 0(w) 

in expectation. Since the size of T is also 0{w)^ step 5 takes 0{w) time if Vk G T, which happens 
with probability at least 1 — 2/n^, and 0{n) otherwise. Similar to the previous step, the expected 
cost is 0{vu). □ 

Note that with a slight alteration to the \BooleanS election] algorithm it is possible to improve 
upon Theorem 1121 if p is much smaller than 1/ logn. Namely, setting 

w = 150p~^ \ogn\og{n/p) , 

and appealing to Lemma |4] in the analysis, gives an expected cost of 

O (p"Mognlog(n/p)) . 

5 Unit and Infinite Comparison Costs 

In this section we consider the setting where only a subset of the comparisons is allowed. More 
specifically, each comparison is allowed with probability p (has cost 1) and is not allowed otherwise 
(has infinite cost). Here, the underlying total order might not be possible to infer even if all 
comparisons are performed. This is because, for example, adjacent elements can be compared only 
with probability p. Hence, even the maximum element might not be possible to certify exactly. 
We therefore relax our goals to finding maximal elements and inferring the poset defined by the 
edges of cost 1. In what follows, we present algorithms for finding a maximal element as well as all 
maximal elements (see Fig. [3]). We consider an element maximal if it wins (directly or indirectly) 
all allowed comparisons to its neighbors. 

Theorem 13. The expected cost of the cheapest certificate for all maximal elements is 

n{nii-ii-pr-')) . 
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Algorithm PosetFindMaximal 

1. Pick veV 

2. while \V\ > 

3. do Perform cost 1 comparisons witli v until it loses (or is certified as max- 

imal) 

4. if V wins all of its comparisons then return v maximal 

5. else V ^ V \ {v} and set v to the winner of the last comparison 

Algorithm PosetFindAllMaximal 

1. for each v & V 

2. do Perform, in a random order, cost 1 comparisons with v 

3. until ti loses or all such comparisons are performed 

4. return All elements that did not lose comparison 



Figure 3: Algorithms for l/oo comparison costs 



Proof. In this setting, each element that has no edges of cost 1 incident to it is a maximal element. 
In expectation, there are n{l — p)"~^ such elements. For each of the remaining elements we need to 
do at least one comparison. Note that each comparison satisfies this requirement for two elements. 
Therefore, we need to do at least ^{n — nil — comparisons in expectation. □ 

Theorem 14. The expected cost of \PosetFindAllMaximcil\ is 0{n\ogn). The expected cost of 
\Pose tFindMaxim al I is at most n — 1. 

Proof. We first analyze \PosetFindAllMaximal\ Fix an element v. Let i = rky{v). Consider the 
following equivalent random process that assigns costs (1 or oo) to edges in the following way: 

1. Pick t from a random variable T distributed as Bin(n — 

2. Repeat t times: Assign cost 1 to a random edge adjacent to v whose cost has not yet been 
determined. 

3. Declare the cost of all other edges adjacent to v to be oo. 

4. For each remaining graph edge assign cost 1 with probability p and oo otherwise. 

We may assume that the algorithm probes the cost 1 edges in this order until v loses a comparison 
or until all cost 1 edges are revealed. If v has not lost a comparison, there is a probability of at 
least [i — l)/(n — 1) that v loses the next performed comparison. Hence, the expected number of 
comparisons involving v is 

E™ r , v-^ i — 1 f i — \ \'' ^ v-^ , ,71—1 n — 1 
t j=i \ ^ t 

Therefore, by linearity of expectation the total number of comparisons we expect to do is bounded 
above by {n — l)iJ„_i + {n ~ 1). 

The second part of the theorem follows easily from Proposition 1. The alsoiithm lPosetFindMaximaU 
is given for completeness. □ 
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6 Conclusions and Open Questions 



We have presented a range of algorithms for finding cheap sorting/selection certificates in three 
different stochastic priced-information models. Most of our algorithms are optimal up to constants 
and the remaining algorithms are optimal up to poly-logarithmic terms (for constant values of the 
parameter p). Beyond improving the existing algorithms there arc numerous ways to extend this 
work. In particular, 

• What about the price model in which the comparison costs are chosen in an adversarial 
manner but the order of the elements is randomized? 

• In this work we have compared expected cost of minimum certificates to expected cost of the 
algorithms presented. Is it possible to design algorithm which are optimal in the sense that 
the expected cost of the certificate found is minimal over all algorithms? Perhaps this would 
admit an information theoretic approach. 

Finally, this work was partially motivated by the game theoretic framework described in Sec- 
tion [TTT] A full treatment of this problem was beyond the scope of the present work. However, the 
problem seems natural and deserving of further investigation. 
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